Skip to content
This repository was archived by the owner on Apr 26, 2026. It is now read-only.

Add image support in user messages#13

Open
Aptul9 wants to merge 1 commit into
unixfox:masterfrom
Aptul9:feat/vision-support
Open

Add image support in user messages#13
Aptul9 wants to merge 1 commit into
unixfox:masterfrom
Aptul9:feat/vision-support

Conversation

@Aptul9

@Aptul9 Aptul9 commented Apr 17, 2026

Copy link
Copy Markdown

Summary

Convert AI SDK v6 file / image parts into the Anthropic image content block ({ type: "image", source: { type: "base64", media_type, data } }) expected by the Claude CLI on stream-json input. Without this change the plugin silently drops attachments and the model replies as if vision were unsupported.

Behavior

Accepts:

  • Base64 data URIs (data:image/png;base64,...) — the format OpenCode stores on disk
  • Raw base64 strings with mediaType / mime / mimeType
  • Uint8Array buffers

Rejects with a warning (returns null, part is skipped):

  • Remote http(s):// URLs — Claude API only accepts inline base64
  • Media types outside image/png, image/jpeg, image/gif, image/webp
  • Missing data / url / source.data

Config requirements

Models that should accept images must declare both in opencode.json:

{
  "attachment": true,
  "modalities": { "input": ["text", "image"], "output": ["text"] }
}

Without modalities.input including image, OpenCode filters file parts before they reach the plugin and replaces them with a [Image #N] text placeholder.

Test plan

  • Roundtrip via Claude CLI stream-json: PNG inline base64 correctly described by Haiku 4.5
  • End-to-end via OpenCode UI (with modalities set): image attached, Sonnet 4.6 and Opus 4.7 describe content correctly
  • Unit checks for buildImageBlock: data URI, raw base64, Uint8Array, http(s) URL (rejected), unsupported media type (rejected), missing data (rejected)

Split from a previous combined PR per maintainer request. Depends on nothing; builds on top of current master.

Convert AI SDK v6 file/image parts into the Anthropic image content
block format expected by the Claude CLI on stream-json input. Accepts
base64 data URIs (the format OpenCode stores on disk), raw base64
strings with mediaType/mime/mimeType, and Uint8Array buffers. Remote
URLs and unsupported media types are skipped with a warning.

Requires the model config in opencode.json to declare
  "modalities": { "input": ["text", "image"], "output": ["text"] }
and "attachment": true, otherwise OpenCode filters file parts before
they reach the plugin.

Also ignore the local context.md scratchpad.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant